Comparing Pure Parallel Ensemble Creation Techniques Against Bagging

We experimentally evaluate bagging and seven other randomization-based approaches to creating an ensemble of decision-tree classifiers. Unlike methods related to boosting, all of the eight approaches create each classifier in an ensemble independently of the other classifiers in the ensemble. Bagging uses randomization to create multiple training sets. Other approaches, such as those of Dietterich, apply randomization in selecting a test at a given node of a tree. Then there are approaches, such as Breiman’s random forests and Ho’s random subspaces, which apply randomization in the selection of attributes to be used in building the tree. Experiments were performed on 28 publicly available datasets, using C4.5 release 8 as the base classifier. While each of the other seven approaches has some strengths, we find that none of them is consistently more accurate than standard bagging when tested for statistical significance.